feat: added scatter summary plot per horizon for all indicators - #753
Conversation
for more information, see https://pre-commit.ci
measrainsey
left a comment
measrainsey
left a comment
There was a problem hiding this comment.
Thank you @AndreasHD11 for implementing this feature! It's really helpful for getting an overview of how our CBA indicators look compared to TYNDP 2024 :)
I can confirm that it works on my end when I run the CBA workflow with run.name = "NT-cyears" and cba.projects: [-t339] (the t339 problem is being fixed in #750), the following files are generated for me:
results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2030.pngresults/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2040.png
Can I make two requests:
- I don't think these summary benchmark plots are generated when running a scenario like
NT(I tested). Of course in these cases we would have less data points to plot, but I think we would want that functionality, since we do a lot of checking results with just the NT scenario and not always with a collection scenario likeNT-cyears. Would it be possible to modify your implementation to work also with single scenarios likeNT,NT-cy2009, etc? - Is it possible to have different diagonal lines for the b2a_societal_cost indicator (one for low/med/high, in the same colors as the dots), instead of one shared diagonal line?
|
I ran it again with the new changes with the following config and commands:
Using results from: NT-cyears-20260425 2040: I also ran it with more recent results and only one climate year such as NT-cy2009: 2030: 2040: I also tested with just NT for run.name and it also worked. |
|
For this: Is it possible to have different diagonal lines for the b2a_societal_cost indicator (one for low/med/high, in the same colors as the dots), instead of one shared diagonal line? The diagonal lines are just comparison y=x lines at 45 degrees. So when a dot (indicator value of project t) is on the line it means the TYNDP and Open-TYNDP values are equal. If it's higher then our values are higher than TYNDP for example. Having a diagonal line for each scenario in b2a would just be 3 superimposed lines I'm guessing. Could you clarify if the question is different to my answer? Thanks! |
lisazeyen
left a comment
lisazeyen
left a comment
There was a problem hiding this comment.
@AndreasHD11 awesome, thanks for preparing! The plots look really nice.
A few points to consider:
add overall indicator
What I am missing is an overall value for each indicator which shows us if we are moving closer to the official TYNDP indicators. I would suggest to
1.) calculate an indicator similar to SB benchmarking plot_benchmark.py
2.) add this value to each plot as a text, similar to here
societal emission costs
I think we can just compare the central level for the benchmarking, the other two levels should be related, since it is a multiplication of costs*emissions, and only the emissions are part of the optimisation
merge conflicts
Merge conflicts need to be resolved.
|
Removing the low and high scenarios (and colours) for b2a, and adding a box with the n, sMAPE and sMdAPE for each indicator: 2030: 2040: |
# Conflicts: # doc/release_notes.rst # scripts/cba/plot_benchmark_indicators.py
for more information, see https://pre-commit.ci
|
Am fixing failed test for cba.smk |
|
The output of the rule initially did not include a run wildcard, but the rule asked for a run wildcard, which means that it worked locally for me as run.scenarios was enabled. But in the test it would fail. |
Meas is on holidays, LZ will review instead
lisazeyen
left a comment
lisazeyen
left a comment
There was a problem hiding this comment.
Great, thanks a lot @AndreasHD11 ! Good to go!
|
Thank you for the feature! One small suggestion, I think it would help to have equal axis on the plots. This would make them easier to read. It's too late for this PR, but it could be added later. |
4 participants
Closes #685 #712
Changes proposed in this Pull Request
This PR adds a scatter plot per planning horizon, for each indicator for all projects. The plot compares TYNDP and Open-tyndp weighted average values for each indicator.
Workflow
To test, I downloaded the NT-cyears-20260425 folder as it contained all the ensemble indicators for each project (cy weighted average values per project).
I ran:
snakemake --configfile config/config.tyndp.yaml --cores 1 "results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2030.png" "results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2040.png" --allowed-rules plot_summary_projects_benchmark --rerun-triggers mtimeIn ensemble_plots folder:
2030
2040
Notes
config/config.tyndp.yaml, I ranrun.name: "NT-cyears". For the projects, I ran incba.projects: t1-t1300(minus t339 as this was creating issues).Changes
cba.smkcalledrule plot_summary_projects_benchmarkplot_benchmark_indicators.pycalledplot_summary_projects_benchmarkChecklist
Required:
doc/release_notes.md.If applicable:
scripts/lib/validation.doc/*.mdfiles.